Toward unconstrained command and control: data-driven semantic inference

Command and control tasks are typically approached using a context-free grammar as language model. While ensuring a good system performance, this imposes a rigid framework on users, by implicitly forcing them to conform to a pre-de ned interaction structure. This paper introduces the concept of data-driven semantic inference, which in principle allows for any word constructs in command/query formulation. Each unconstrained word string is automatically mapped onto the intended action through a semantic classi cation against the set of supported actions. The underlying (latent semantic analysis) framework relies on co-occurrences between words and commands, as observed in a training corpus. A suitable extension can also handle commands that are ambiguous at the word level. Experiments conducted on a desktop command and control task involving 113 di erent actions show a classi cation error rate of 1.7%.